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Abstract 

Several preconditioned AOR methods have been proposed to solve system of 
linear equations Ax = b, where A £ M. nxn is a unit Z-matrix. The aim of this paper 
is to give a comparison result for a class of preconditioners P, where P £ M. nxn is 
nonsingular, nonnegative and has unit diagonal entries. Numerical results for cor- 
responding preconditioned GMRES methods are given to illustrate the theoretical 
results. 
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1. Introduction 

Consider the system of linear equations 

Ax = b, (1) 

where A = (a^) £ ~R nxn and b £ R n . A general stationary iterative method for 
solving Eq. (JTJ may be expressed as 



x 



= M^Nx^ +M~ 1 b, k = 0,1,2, 



in which A = M — N, where M, N £ K nx " and M is nonsingular. It is well-known 
that this iterative method is convergent if and only if p(M~ 1 N) < 1, where p(X) 
refers to the spectral radius of matrix X. Let a„ ^ 0, i — 1, 2, . . . , n. Therefore, 
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without loss of generality we can assume that an = 1, i — 1, 2, . . . , n. In this case, 
we split A into 

A — I — L — U, (2) 

where / is the identity matrix, — L and —U are strictly lower and strictly upper 
triangular matrices, respectively. The accelerated overrelaxation (AOR) iterative 
method to solve Eq. ([T]) is defined by 0, 12 



x (k+1) = C^x (k) + w(J - TL)- 1 ^ 



in which 

= (I - 7^) _1 [(1 - w)I + (w - l)L + wtf], 

where w and 7 are real parameters and with u> 7^ 0. For certain values of the 
parameters u and 7 the AOR iterative method results in the Jacobi, Gauss-Seidel 
and the SOR methods (j. 

To improve the convergence rate of an iterative method one may apply it to the 
preconditioned linear system PAx = Pb, where the matrix P is called a precon- 
ditioned Several preconditioners have been presented for the stationary iterative 



methods by many authors. Recently, Wang and Song in [13| have proposed a 
general preconditioner P which is nonsingular, nonnegative and has unit diagonal 
entries. They also have investigated the properties of the preconditioners of the 
form 

p = (Pij) = (-ayOti), (3) 

where < < 1, for i ^ j, and pa = 1 for i — 1, . . . , n. Many preconditioners 
proposed in the literature are the special cases of such general preconditioner (see 

for example @, i H H B B EE Q)- in this paper, we show that under some 



conditions, preconditioner with = 1 for 1 < i 7^ j < n is the best one among 
the preconditioners of the form ([3]). 

For convenience, we first present some notations, definitions and preliminaries 
which will be used in this paper. A matrix A = (ay) G ]R mxn is said to be 
nonnegative and denoted by A > if ay > for all % and j and A is said to 
positive and denoted by A ^> if ay > for all % and j. If A > 0, then by 
the Perron- Frobenius theory (see for example [2|), p(A) is an eigenvalue of A, and 
corresponding to p(A), A has a nonnegative eigenvector, which we refer to as a 
Perron vector of A. 

Definition 1.1. A matrix A = (ay) £ M nxra is an Z-matrix if ay < for z 7^ j. 

Definition 1.2. A Z-matrix A is said to be an M-matrix if A is nonsingular and 
A- 1 > 0. 
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Definition 1.3. Let A e M nxn . The representation A = M — N is called a splitting 
of A if M is nonsingular. The splitting A = M — N is called 

(a) convergent if p(M~ l N) < 1; 

(b) weaA; reow/ar if M -1 > and M~ l N > 0; 

(c) an M- splitting of A if M is an M-matrix and N > 0. 

Definition 1.4. A real matrix A is called monotone if Ax > implies x > 0. 

Lemma 1.1. j^, Lemma 3.2 ] Let A = M — N be an M-splitting of A. Then 
p(M~ 1 N) < 1 if and only if A is an M-matrix. 



Lemma 1.2. [l_3j, Lemma 2.2 1 Let A = (a^-) e M nxn 6e an M-matrix. Then, there 



exists e > such that, for any < e < e , A(e) = (ay(e)) is aiso an M-matrix, 
where 

ctij, if Oij ^ 0, 
-e, z/ a,ij = 0. 



Lemma 1.3. Lemma 6.1 ] A zs monotone if and only if A is nonsingular with 
A' 1 > 0. 



Lemma 1.4. 1% Lemma 1.6 ] Let A be a Z-matrix. Then, A is an M-matrix if 



and only if there is a positive vector x such that Ax 3> 0. 

Lemma 1.5. (lol . Lemma 2.2] Suppose that A\ = Mi — Ni and A 2 = M 2 — N 2 are 

weak regular splittings of the monotone matrices A\ and A 2 , respectively, such that 
M 2 l > Mi . If there exists a positive vector x such that < Aix < A 2 x, then for 
the monotonic norm associated with x, 

\\M 2 l N 2 \\ x < HM^JVxIU. 

In particular, if M^Ni has a positive Perron vector, then 

p{M^ l N 2 ) < p(Mf x iV 2 ). 



2. Main results 

Let A e M nxn . We consider the preconditioner P = (p^) for Eq. ([T]), where 



Pij 



-aijdij, if i ^ j, 
1, otherwise, 



and ay G M for % ^ j. We split P into P = I + + C/(a;), where / is the 
identity matrix and L(a) and U(a) are strictly lower and strictly upper triangular 
matrices, respectively Let A = PA = (I + L(a) + U(a))A and 

L(a)U = Ga(a) + E 1 {a) + Fi(a), 
tf(a)L = G 2 {a) + £ 2 (a) + F 2 {a), 

where Gi(a) and G2(a) are diagonal matrices, Fi(a) and F 2 (a) are strictly lower 
triangular matrices and Ei(a) and i?2(«) are strictly upper triangular matrices. In 
this case, A can be split as A = D — L — U, where D, L and U, respectively, are 
diagonal, strictly lower and strictly upper triangular matrices defined as 

D = I-Exia) -E 2 {a), 

L = L- L(a) + L(a)L + F^a) + F 2 {a), 

U = U + G x {a) - U(a) + G 2 (a) + U(a)U. 

If the matrix D — rL is nonsingular, the AOR iteration matrix to solve the precon- 
ditioned system PAx = Pb can be written as 

= {D- jL)- l [(l - co)D + {u>- j)L + uU}. 



Theorem 2.1. Let A be a Z-matrix and G [0, 1] for 1 < i ^ j < n. Then, A is 
M-matrix if and only if A is and M-matrix. 



an 



Proof. The proof of this theorem is similar to the proof of Lemma 3.3 in [9j. Let 



A be an M-matrix and A = Then 



a ij 



n 



22 a ikdikaki, 1 <i = j <n, 

(4) 

22 a ik a ik a kj , 1 < % ^ j < n. 



n 



Since A is a Z-matrix, we have o»j < 0, for i ^ j. This means that A is also a 
Z-matrix. By Lemma 1.4 there exists a positive vector x such that Ax 3> 0. On 
the other hand, A = (I + L(a) + U(a))Ax ^> 0. Invoking Lemma 1.4 implies that 
A is also an M-matrix. 

Conversely, let A be an M-matrix. Then, A T is also an M-matrix. By Lemma 1.4 
there exists a positive vector x such that A T x 3> 0, i.e., A T (I + L(a) T + U \a) T )x ^> 
0. Let y = (I + L(a) T + U(a) T )x. Obviously y 3> 0. Therefore by Lemma 1.4, A T 
is an M-matrix. As a result, A is an M-matrix, as well. □ 
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Theorem 2.2. Let A = (ay) G M nxn 6e a nonsingular Z-matrix, < 7 < to < 1, 
a; 7^ and G [0, 1] /or 1 < i 7^ j < n. 7/p(L 7)W ) < 1, then p(£ 7>aj ) < p(£ 7)W ) < 
1. 

Proof. Under the assumptions of the theorem, it is easy to see that 

A = M — N = —(I - jL) [(1 - oo)I + {co- j)L + 00U), 

00 to 

is an M-splitting of A. On the other hand, we have p(M~ 1 A r ) = p(£ 7iW ) < 1. 
Therefore, from Lemma 1.1 we deduce that A is an M-matrix. Now, the result 
follows immediately from Theorems 2.6 and 2.7 in ljjj]. □ 



In the sequel, we show that among the preconditioners of the form P = I + 
L(a) + U(a) with ctij G [0, 1], the preconditioner P = I + L + U is the best one 
to speed up the convergence rate of the AOR iterative method. We mention that, 
if we assume = 1, for 1 < ! / j < n, then the preconditioner P results in 
the preconditioner P. Let the AOR iteration matrix of the preconditioned system 
PAx = Pb be 

4,- = (D~ 7^) _1 [(1 " u)D + (00 - i)L + 10U], 

where A = PA = D — L — U in which D, L and U are the diagonal, strictly lower 
and strictly upper triangular matrices, respectively. 

Theorem 2.3. Let A = (a^) G W nxn be a nonsingular Z-matrix. Let also < 7 < 
co < 1, 00 7^ and G [0, 1] for 1 < i 7^ j < n. If p(£ 7jW ) < 1 and 



n n i— 1 i—X 

— l)Ojj + ( <^ik a ik a kj — 0-ik a kj) + C^ J °^ik a ik a kj — '^^ a ik a >kj) < 0, 

fe=l,fc^i k=l,k^i k=l k=l 



for 1 < j < i < n, then p(£ 7jW ) < p(£ 7iW ). 
Proof. Let 



M = 


V 

oo 


-JL), 




AT = 


> 


-co)I + 


{lo - j)L + coU], 


M = 


OO 






N = 


-oo)b + 
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M=-(D- 1 L), 

cu 

N — — [(1 — u)D + (ou- 7 )L + wE/]. 

It is easy to see that 

.1 M N, A = M-N, A = M-N. 

Similar to the proof of Theorem 2.2 we see that the matrix A is an M-matrix. Let 
x = A~ l e, where e = (1, . . . , 1) T . We have x 3> 0, since none of the rows of A" 1 
can be zero. Therefore, 

(A - A)x = [(L - L{a)) + (U - U{a))]Ax = [(L - L(a)) + (U - U{a))]e > 0. 

Since p(£ 7A ,) < 1, from Theorem 2.2 we have p(L 1>w ) < p(C lt J) < 1. Now, Theo- 
rem 2.1 shows that A is an M-matrix. Hence, D is a diagonal matrix with positive 
diagonal entries. Therefore, M and M are nonsingular matrices and splittings 
A = M - N and A = M - N are M-splitting. 
By definition of A and A, we have 

D-D = E^l) - E ± (a) + E 2 (l) - E 2 (a) > 0. 

So D > D. On the other hand, we have 

L-L = (L- L(a)) + (L(a) - L)L + (F 2 (a) - F 2 (l)) + (F 1 (a) - F^l)). 

Hence, 

i-l i-l 

(Z — L) i;? - = (aiij — l)ajj + a ik a ik a k j — ^ dik^kj) 

k=l k=l 
n n i—l i—l 

+ ( ^ cn ik a ik a k j — ^ a-ikO-kj) + C^OLikdikO-kj — QifcQfcj) 

fc=i+l fc=i+l fc=l fc=l 

n n 

= (a^ — l)aij + ( ^ oz ik a ik a k j — ^ a-ikO-kj) 

k=l,k^i k=l,k^i 
i-l i-l 

Jr C^ a ik a ikO'kj — ^^^ik^kj) < 0, 
fc=l fc=l 

which shows that L < L. This result together with D > D gives l) — 7L < D — ^L. 
Since r yD~ l L > is an strictly lower triangular matrix and p^D^L) < 1, we 
deduce that 

00 

(L> - 7L)- 1 = (/ - -fD^L)- 1 !)- 1 =1 + Y^ilD^Lyb- 1 > 0. 
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In the same way, (D — jL) 1 > 0. Therefore, we obtain 

(5- 7 L)- 1 <(5- T L)- 1 , 

and this means that 

< M- 1 < M~ x . 

Let A be an irreducible matrix. Having in mind that the entries of A are given by 
(jlD, we conclude that A is also an irreducible matrix. We have 

£ 7 ,u, = {D- iL)~ l [{l - u)D + {w- i)L + tuU] 

= (I- 7Z)- 1 L)- 1 [(1 - a) I + (w - 7)£>~ 1 L + wZT 1 ^] 

= [ / + ( 7j D _1 ^) + (jD^L) 2 + ■ ■ ■ ][(1 - + (w - t)^ -1 ^ + w-D -1 ^ 

> [(1 - w)I + (w - tJzHZ + wfl" 1 ^. 

This shows that for every < 7 < 1 the matrix £ 7iW is a nonnegative irreducible 
matrix. Hence, from Theorem 4.11 in |l|, £7,^ = M~ l N has a positive Perron 
vector and from Lemma 1.5 we have 

If 7 = 1, then cu = 7 = 1 and we have 

= lim p(£ 7i i) > lim p(£ lt i) = /o(£i,i). 
7— >i- 7— »i _ 

Now, if A is a reducible matrix, then by Lemma 1.2, for sufficiently e > the 
matrix A(e) is an irreducible M- matrix and one can see that 

p(£ 7 ,J = lim l p(£ 7iW (e)) > lim i ^(e)) = p{C 1>w ). □ 



3. Numerical experiments 

All the numerical experiments presented in this section were computed in double 
precision with some MATLAB codes on a Pentium 4 PC, with a 3.06 GHz CPU 
and 1.00GB of RAM. 

Example 1. We consider the two dimensional convection-diffusion equation (see 

3) 

- {U XX + Uyy) + U X + 2Uy = f (x , lj) , HI ft = (0, l) X (0, l), (5) 

with the homogeneous Dirichlet boundary conditions. Discretization of this equa- 
tion on a uniform grid with N x N interior nodes (n = N 2 ), by using the second 
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order centered differences for the second and first order differentials gives a linear 
system of equations of order n with n unknowns. The coefficient matrix of the 
obtained system is of the form 

A = I®P + Q®I, 
where £g> denotes the Kronecker product, 

, / 2 + h 2-/i. , n , 1+h 
P = tndiag( — ,1, — ) and Q = tridiag( — , 0, — ), 

are N x N tridiagonal matrices, and the step size is h = 1/N. We consider four 
preconditioners of the form 

Po = I, 

P 1 = 1 + 0.5L, 

P 2 = I + 0.5L + 0.5P, 

P 3 = I + L(a) + U(a), 

P 4 = I + L + U, 

where for the preconditioner P3, aiy's are random numbers uniformly distributed 
in the interval (0,1). We mention that Pq = I means that no preconditioner is 
used. In Table 1, the spectral radius of the AOR iterative method applied to the 
preconditioned systems PiAx = PJb, i — 0, . . . , 4 for different values of 7, uj and 
n are given. As we observe preconditioner P4 is the best one among the chosen 
preconditioners. 

For more investigation, we apply the GMRES(m) method [llf with m = 10 to 
solve PiAx = Pib, i = 0, . . . , 4. In all the experiments, vector b = A(l, 1, . . . , 1) T 
was taken to be the right-hand side of the linear system and a null vector as an 
initial guess. The stopping criterion used was always 



I b — Ax 



< 10" 
II&II2 

In Table 2, we report the number of iterations and the CPU time (in parenthesis) 
for the convergence. As we see the preconditioner P 4 is the best. 

Example 2. We consider the previous example with 

-{u xx + u yy ) + 2e x+y {xu x + yu y ) = f(x,y), in Q = (0, 1) x (0, 1). 

All of the assumptions are the same as the previous example. In Table 3 the 
spectral radii of the AOR iterative method and in Table 4 numerical results of 
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Tabic 1: Comparison of spectral radii for Example 1. 



N 


(7»w) 


Po 


Pi 


P 2 


Ps 


Pa 


5 


(0.7,0.8) 


0.8317 


0.7964 


0.7404 


0.7486 


0.6323 


5 


(0.8,1) 


0.7739 


0.7305 


0.6540 


0.6444 


0.5138 


10 


(0.7,0.8) 


0.9474 


0.9350 


0.9125 


0.9116 


0.8677 


10 


(0.8,1) 


0.9289 


0.9135 


0.8821 


0.8815 


0.8221 



Table 2: Number of iterations and the CPU time for the convergence of the GMRES(IO) for 
Example 1. 



N 


^0 


Pi 


Pi 


^3 


Pa 


50 

100 

150 


80(0.34) 
326(5.33) 
702(29.08) 


57 (0.31) 
130(2.83) 
365(19.73) 


33(0.22) 
132(3.30) 
244(15.56) 


44(0.74) 
110(2.75) 
350(23.02) 


29(0.17) 
78 (1.97) 
185(12.08) 



the GMRES(IO) method applied to the preconditioned systems PiAx = Pjb, i = 
0,...,4 are given. As we observe preconditioner P 4 is the best one among the 
chosen preconditioners. 

4. Conclusion 

For a class of matrices, we have shown that among the preconditioners of the 
form P = I + L(a) +U(a) with aiy e [0, 1], the preconditioner P = I + L + U is the 
best one to speed up the convergence rate of the AOR iterative method. Numerical 
results of the AOR and GMRES(m) methods applied to different preconditioned 
systems confirm the presented theoretical results. 



Table 3: Comparison of spectral radii for Example 2. 



N 


(7,^) 


Po 


Pi 


P 2 


Ps 


Pa 


5 


(0.7,0.8) 


0.8657 


0.8358 


0.7871 


0.8049 


0.6907 


5 


(0.8,1) 


0.8193 


0.7823 


0.7154 


0.7033 


0.5891 


10 


(0.7,0.8) 


0.9581 


0.9481 


0.9298 


0.9306 


0.8929 


10 


(0.8,1) 


0.9434 


0.9309 


0.9053 


0.9045 


0.8558 
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Table 4: Number of iterations and the CPU time for the convergence of the GMRES(IO) for 
Example 2. 



N 


Po 


Pi 


P2 


P 3 


Pa 


50 


57(0.27) 


46 (0.27) 


28(0.19) 


36 (0.22) 


23(0.13) 


60 


85(0.52) 


57 (0.45) 


34(0.31) 


49 (0.44) 


29(0.25) 


70 


92(0.72) 


79(0.89) 


49(0.61) 


54 (0.72) 


37(0.47) 


80 


111(1.23) 


84(1.23) 


52(0.84) 


94 (1.55) 


45(0.73) 
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